IRWIN AND JOAN JACOBS CENTER FOR COMMUNICATION AND INFORMATION TECHNOLOGIES Discrete-Input Two-Dimensional Gaussian Channels with Memory: Estimation and Information Rates via Graphical Models and Statistical Mechanics

Manuscript received September 18, 2006; revised September 18, 2007. The material in this paper was presented in part at the IEEE Information Theory Workshop (ITW), San-Antonio, Texas, USA, October 2004, and in the IEEE International Symposium on Information Theory (ISIT), Adelaide, Australia, September 2005. O. Shental was with the Department of Electrical Engineering-Systems, Tel Aviv University, Tel Aviv 69978, Israel. He is now with the Center for Magnetic Recording Research (CMRR), University of California San Diego (UCSD), 9500 Gilman Drive, La Jolla, CA 92093, USA (e-mail: [email protected]). N. Shental is with the Department of Physics of Complex Systems, Weizmann Institute of Science, Rehovot 76100, Israel (e-mail: [email protected]). S. Shamai (Shitz) is with the Department of Electrical Engineering, Technion-Israel Institute of Technology, Haifa 32000, Israel (e-mail: [email protected]). I. Kanter is with the Minerva Center and Department of Physics, Bar-Ilan University, Ramat-Gan 52900, Israel (e-mail: [email protected]). A. J. Weiss is with the Department of Electrical Engineering-Systems, Tel Aviv University, Tel Aviv 69978, Israel (e-mail: [email protected]). Y. Weiss is with the School of Computer Science and Engineering, Center for Neural Computation, Hebrew University of Jerusalem, Jerusalem 91904, Israel (e-mail: [email protected]). The first two authors contributed equally to this work. Communicated by G. Kramer, Associate Editor for Shannon Theory. IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. XX, NO. Y, MONTH 200Z 2 Discrete-input two-dimensional (2-D) Gaussian channels with memory represent an important class of systems, which appears extensively in communications and storage. In spite of their widespread use, the workings of 2-D channels are still very much unknown. In this work we try to explore their properties from the perspective of estimation theory and information theory. At the heart of our approach is a mapping of a 2-D channel to an undirected graphical model, and inferring its a-posteriori probabilities using generalized belief propagation (GBP). The derived probabilities are shown to be practically accurate, thus enabling optimal maximum a-posteriori (MAP) estimation of the transmitted symbols. Also, the Shannon-theoretic information rates are deduced either via the vector-wise Shannon-McMillan-Breiman theorem, or via the recently derived symbol-wise GuoShamai-Verdú theorem. Our approach is also described from the perspective of statistical mechanics, as the graphical model and inference algorithm have their analogues in physics. Our experimental study, based on common channel settings taken from cellular networks and magnetic recording devices, demonstrates that under non-trivial memory conditions, the performance of this fully tractable GBP estimator is almost identical to the performance of the optimal MAP estimator. It also enables a practically accurate simulation-based estimate of the information rate. Rationalization of this excellent performance of GBP in the 2-D Gaussian channel setting is addressed.